Head Mounted Displays (HMDs) are becoming increasingly popular for augmented reality (AR) and virtual reality (VR) applications. While many AR systems provide “see-through” transparent or translucent displays upon which to project virtual objects, many VR systems instead employ opaque, enclosed screens. These enclosed screens may completely obscure the user's field of view of the real world. Indeed, eliminating the user's perception of the real world may be integral to a successful VR experience.
HMDs designed exclusively for VR experiences may fail to capture significant portions of the AR market. For instance, despite possibly including functionality for capturing and presenting images of the user's real-world field of view, VR headsets may still not readily lend themselves to being repurposed for AR applications. Accordingly, it may be desirable to allow users to repurpose a VR HMD for use as an AR device. Alternatively, one may simply wish to design an AR device that does not incorporate a transparent or translucent real-world field of view to the user. Such HMDs may already include a camera and/or pose estimation system as part of their original functionality, e.g., as described in U.S. Provisional Patent Application 62/080,400 and U.S. Provisional Patent Application 62/080,983. For example, an immersive VR experience may rely upon an inertial measurement unit (IMU), electromagnetic transponders, laser-based range-finder systems, depth-data based localization with a previously captured environment model, etc. to determine the location and orientation of the HMD, and consequently, the user's head. Accordingly, the disclosed embodiments provide AR functionality for opaque, “non-see-through” HMDs (generally referred to as a VR HMD herein), which may include, e.g., an RGB or RGBD camera.
While the flow and sequence diagrams presented herein show an organization designed to make them more comprehensible by a human reader, those skilled in the art will appreciate that actual data structures used to store this information may differ from what is shown, in that they, for example, may be organized in a different manner; may contain more or less information than shown; may be compressed and/or encrypted; etc.
The headings provided herein are for convenience only and do not necessarily affect the scope or meaning of the embodiments. Further, the drawings have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be expanded or reduced to help improve the understanding of the embodiments. Similarly, some components and/or operations may be separated into different blocks or combined into a single block for the purposes of discussion of some of the embodiments. Moreover, while the various embodiments are amenable to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are described in detail below. The intention, however, is not to limit the particular embodiments described. On the contrary, the embodiments are intended to cover all modifications, equivalents, and alternatives falling within the scope of the disclosed embodiments.